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Executive  Summary 

The  Battlescale  Forecast  Model  (BFM)  is  statistically  evaluated  by  comparing 
*  the  forecast  data  to  observed  data.  BFM  was  run  for  cases  of  a  30-day 

period  over  three  different  model  domains  having  different  terrain 
t  complexities  and  climate  (Colorado,  Washington  and  Florida).  The  model 

calculations  were  initialized  with  three  different  sets  of  initial  conditions: 

(1)  Navy  Operational  Global  Atmospheric  Prediction  System 
(NOGAPS)  +  upper  air  +  surface  data, 

(2)  NOGAPS  +  upper  air,  and 

(3)  NOGAPS. 

Forecast  data  for  24-h  periods  are  statistically  compared  with  surface 
observation  data,  by  calculating  parameters  such  as  mean  difference  (MD), 
absolute  difference  (AD),  root  mean  square  error,  and  root  mean  square 
vector  error. 

For  all  three  model  domains,  the  temperature  fields  of  BFM  initialized  with 
(1)  and  (2)  above  are  statistically  better  than  those  initialized  with  (3).  For 
Colorado  and  Washington  model  domains,  the  BFM  showed  clear 
tendencies  of  forecasting  dew  point  temperatures  lower  than  those  observed 
throughout  the  24-h  forecast  period.  However  for  the  Florida  model 
domain,  forecasts  of  dew  point  temperature  are  higher  than  observed. 

Three  different  types  of  initialization  data  did  not  produce  significantly 
different  wind  fields  throughout  the  24-h  forecast  period.  The  values  of  MD 
for  wind  speed  are  in  the  range  of  0  to  1  m/ sec,  and  those  of  AD  are  also 
between  0  and  1  m/sec  for  three  model  domains  throughout  the  24-h 
forecast  period. 

For  Colorado  and  Washington  model  domains  where  terrain  is  more 
complex  than  Florida,  the  use  of  the  BFM  improves  temperature  forecasts 
over  those  of  NOGAPS  and  NORAPS.  For  the  Florida  domain,  no 
significant  improvements  in  temperature  forecasts  are  found.  Similarly,  the 
f  BFM  produces  better  wind  fields  than  NOGAPS  and  Navy  Operational 

Regional  Atmospheric  Prediction  System  (NORAPS)  over  Colorado  and 
Washington. 
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1.0  Introduction 

The  Battlescale  Forecast  Model  (BFM)  is  an  operational  mesoscale  forecast 
model,  developed  at  the  U.S.  Army  Research  Laboratory  (ARL).  It  has  been 
extensively  used  to  make  short-range  forecasts  of  atmospheric  conditions  as 
a  component  in  both  the  Integrated  Meteorology  System  (IMETS)  and  the 
Computer  Assisted  Artillery  Meteorology  (CAAM)  System.  The  BFM  uses, 
for  prognostic  calculation,  the  Higher  Order  Turbulence  Model  for 
Atmospheric  Circulation  (HOTMAC)  developed  by  Yamada.  Recently,  the 
forecast  skill  of  the  BFM  was  compared  with  that  of  the  Fifth-Generation 
National  Center  Atmospheric  Research/Penn  State  Mesoscale  Model  (MM5), 
by  applying  the  models  to  the  domain  of  White  Sands  Missile  Range 
(WSMR),  NM,  which  covers  an  area  size  of  167  x  167  km  (51  x51  grid  points 
with  grid  spacing  of  3.33  km).  Meteorological  parameters  forecasted  by  the 
models  were  compared  with  observed  data.  The  comparison  study  showed 
that  the  forecast  skills  of  the  BFM  are  comparable  to  those  of  the 
MM5 — surface  temperature  forecasted  by  both  the  BFM  and  the  MM5  agreed 
well  with  observed  values.  Both  the  BFM  and  the  MM5  showed  difficulties 
forecasting  the  relative  humidity.  For  wind  parameters,  both  models  tend  to 
predict  wind  speed  less  than  observation,  but  the  BFM  calculations  produce 
smaller  wind  speed  than  the  MM5.  For  wind-direction  forecast,  the  BFM 
resulted  in  better  forecast  than  the  MM5.  [1] 

The  BFM  in  operational  mode  on  the  IMETS  has  been  extensively  used  over 
the  model  domain  of  500  x  500  km  with  grid  spacing  of  10  km.  So  far  there 
has  been  no  evaluation  study  of  the  BFM  in  operational  mode,  applying  the 
model  for  an  extended  period  and  comparing  model  forecasted  parameters 
with  observations.  In  this  study,  the  BFM  was  applied  for  30-day  periods  to 
three  different  model  domains,  each  having  an  area  size  of  500  x  500  km  with 
51  x  51  grid  points  and  10  km  grid  spacing. 

The  first  purpose  of  the  present  report  is  to  describe  the  results  of  evaluation 
studies  of  BFM  forecast  skill  over  three  different  areas  with  different  terrain 
complexities  and  climates  (Colorado,  Washington,  and  Florida).  In  this 
study,  the  model  calculations  were  initialized  with  three  different  sets  of 
initial  conditions  of  varying  completeness,  and  model  values  were 
statistically  compared  with  observed  data.  The  second  purpose  is  to 
compare  the  statistics  of  BFM  forecast  data  to  those  of  other  models,  such  as 
the  Navy  Operational  Global  Atmospheric  Prediction  System  (NOGAPS) 
and  the  Navy  Operational  Regional  Atmospheric  Prediction  System 
(NORAPS).  [2,3,4] 
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2.0  Model  Domains 

The  following  three  model  domains  were  chosen  for  the  present  study: 

1.  Colorado,  centered  at  38.8°N  and  104.7°W, 

2.  Washington,  centered  at  46.8°N  and  121.5°W,  and 

3.  Florida,  centered  at  28.0°N  and  81.0°W. 

Each  model  domain  covers  500  x  500  km  area  with  horizontal  grid  spacing 
of  10  km  and  the  horizontal  grid  number  of  51  x  51.  The  vertical  depth  of 
model  is  7  km  above  the  highest  point  of  the  model  domain  with  16  vertical 
layers. 

The  complexities  of  the  three  model  domains  are  summarized  in  table  1. 


Table  1.  Terrain  complexities  of  the  three  model  domains 


Colorado 

Washington 

Florida 

Mean  elevation  (m) 

2022 

590 

7 

Max.  height  (m) 

4053 

4105 

69 

Min.  height  (m) 

1001 

0 

0 

Standard  Dev.  (m) 

722 

526 

12 

As  seen  in  table  1,  Colorado  and  Washington  model  domains  are  complex 
compared  to  the  Florida  model  domain.  Figures  1, 2,  and  3  are  the  elevation 
contour  maps  of,  respectively,  Colorado,  Washington,  and  Florida  model 
domains.  In  these  figures,  the  outside  squares  cover  the  NOGAPS  analysis 
and  data  composite  area  of  800  x  800  km,  and  the  inside  square  covers  the 
model  domain  of  500  x  500  km.  In  these  figures,  the  characters  N,  U,  and  S 
represent  the  locations  of  the  input  data  used  for  initialization  of  the  model 
calculation: 


N  -  NOGAPS  data, 

U  -  upper-air  sounding  data,  and 
S  -  surface  observation  data. 
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Figure  2.  Elevation  contours  of  the  Washington  model  domain. 

Note:  The  outside  area  covers  800  x  800  km,  and  the  inside  area  covers  500  x 
500  km. 

N  =  NOGAPS  data  points, 

U  =  upper-air  sounding  locations,  and 
S  =  surface  observation  locations. 
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Figure  3.  Elevation  contours  of  the  Florida  model  domain. 


Note:  The  outside  area  covers  800  x  800  km,  and  the  inside  area  covers  500  x 
500  km. 

N  =  NOGAPS  data  points, 

U  =  upper-air  sounding  locations,  and 
S  =  surface  observation  locations. 


3.0  Method 


3.1  Data  Used 

r 

The  following  data  sets  were  obtained  from  the  period  of  June  23  to 
July  22,  1998  from  the  indicated  Internet  home  pages.  NOGAPS  and 

*  NORAPS  data  used  for  this  study  were  obtained  through  the  Internet  home 

page  of  the  Master  Environmental  Library  (MEL)  of  the  Defense  Modeling 
and  Simulation  Office  (DMSO)  at:  http:  /  7mel.dmso.mil. 

NOGAPS  forecast  data  were  available  at  every  1°  grid  points  over  the  entire 
earth,  and  NORAPS  forecast  data  were  available  at  every  0.5  °  grid  points 
over  the  North  America  region.  NOGAPS  data  were  used  for  initialization 
and  time-dependent  lateral  boundary  conditions  of  BFM  and,  for  comparison 
with  BFM  forecast  data.  The  forecast  data  for  the  periods  of  12, 24,  and  36  h, 
initialized  at  0000  UTC,  were  obtained  at  13  pressure  levels  from  1000  mb  to 
100  mb.  NORAPS  data  were  used  for  statistical  comparison  with  the  BFM. 

Upper-air  sounding  data  are  obtained  from  the  University  of  Wyoming  at  the 
following  Internet  home  page:  http:  /  7www-das.uwyo.edu /upperair. 
Upper-air  data  observed  at  1200  UTC  were  used  for  this  study.  Surface 
observation  data  were  obtained  from  Ohio  State  University  at  the  following 
Internet  home  page:  http:  /  /geografl.sbs.ohio-state.edu.  The  data  are 
archived  for  00, 03, 09, 12, 15, 18,  and  21  UTC. 

3.2  Initialization  Methods 

For  all  three  model  domains,  BFM  is  initialized  at  12  UTC  with  three 
different  data  sets  as  mentioned  below,  and  forecasted  data  for  24-h  period 
are  compared  with  surface  observation  data: 

(1)  NOGAPS  +  upper  air  +  surface  data, 

(2)  NOGAPS  +  upper  air  data,  and 

(3)  NOGAPS  data  only. 

Details  of  the  initialization  method  of  BFM  are  described  in  reference  1. 
Briefly,  for  the  three  data  sets,  the  1  °  NOGAPS  data  valid  for  the  forecast 
periods  of  0, 12,  and  24  h  are  spatially  interpolated  for  the  outside  square 

*  areas  of  800  x  800  km  (see  figures  1, 2,  and  3).  For  initializations  (1)  and  (2), 

the  interpolated  data  from  NOGAPS  for  the  period  of  0  h  are  composited 
with  upper-air  sounding  data.  For  initialization  (1),  surface  data  of  wind 
vector  components,  temperature,  and  mixing  ratio  at  the  0-h  period  are 
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assimilated  into  the  model  field  by  a  nudging  method.  The  nudging  is  done 
for  3  h  from  T0  -3  to  T0/  where  T0  is  the  start  of  the  forecast  period.  For  the 
present  study,  T0  is  12  UTC  for  all  three  model  domains.  [1,2] 

3.3  Statistical  Parameters 

The  following  verification  statistics  were  calculated  using  the  data  for  the  30 
day  period  between  June  23  and  July  22,  1999.  Temperature,  dew  point  * 

temperature,  wind  speed,  and  vector  components  at  the  surface  were 
compared  by  using  these  parameters  at  the  forecast  periods  of  0, 3, 6, 9, 12, 

15, 21,  and  24  h. 

3.3.1  Mean  Difference 


MD  - 


m  n 

S  ( Xp,i,j  Xo,i,j  ) 


tL  M _ 

m-n 


(i) 


Where 


x  =  arbitrary  meteorological  parameter, 
0  =  observation, 
p  =  prediction, 
i  =  ith  surface  station, 
j  =  3th  forecast  day, 
n  =  surface  stations, 
m  =  total  number  of  forecast  days,  and 
Nonzero  =  MD  indicates  bias. 


For  instance,  if  the  mean  difference  (MD)  is  positive,  it  indicates  that  the 
model  tends  to  overforecast. 


3.3.2  Mean  Absolute  Difference 


AD  = 


m-n 


(2) 


f 


% 
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Because  the  absolute  value  of  the  difference  is  used,  good  agreements 
between  observation  and  forecast  are,  in  general,  related  to  small  values  of 
absolute  difference  (AD). 

3.3.3  Root  Mean  Square  Error 


♦ 


rmse  = 


1 


£  £  ( Xo,i,j  Xp,i,j  ) 

M  i= 1 _ 


m  ■  n 


(3) 


Usually,  the  values  of  root  mean  square  error  (RMSE)  is  proportional  to  those 
of  AD.  [2] 


3.3.4  Root  Mean  Square  Vector  Error 


RMSVE  = 


=\ 


mn 

£  £  [K/j  -  uP,uf + -  vP,uf  ] 


(4) 


J- i  >= i 


m-n 


This  parameter  measures  the  difference  of  both  the  n-s  and  e-w  components 
of  the  wind  vector  giving  a  result  that  effectively  combines  wind  speed  and 
direction.  Again,  good  agreements  of  wind  vectors  are  related  to  small 
values  of  the  root  mean  square  vector  error  (RMSVE). 


3.3.5  Correlation  Coefficient 


CC  = 


£  £  y o,ij '  y pjj 
M_M _ 


ZlAu-ZZrV/ 

>1  <■=!  M  M 


(5) 


Here  yo  i  J  =  xo  iJ  -  xo  ,and  y piJ  =  xp  i  j  -xp  .  xo  and  are  the 
means  of  observed  and  forecast  values,  respectively. 
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Scatter  Diagrams 

The  forecast  data  generated  by  the  BFM  runs  using  the  NOGAPS  +  upper-air 
data  (2)  are  used  for  the  plotting.  Scatter  diagrams  obtained  for  the  model 
domains  of  Colorado,  Washington,  and  Florida  are  shown  in  figures  4, 5,  and 
6,  respectively.  In  these  figures,  (a)  is  for  temperature,  (b)  for  dew  point 
temperature,  (c)  for  wind  vector  components  u  and  v,  and  (d)  for  wind 
speed.  The  correlation  coefficients  between  forecasted  and  observed  data  are 
summarize  in  table  2. 


Table  2.  Correlation  coefficients  between  forecast  and  observation  for  three 
model  domains  of  Colorado,  Washington,  and  Florida 


Colorado 

Washington 

Florida 

Temperature 

0.83 

0.77 

0.65 

Dew  point  temp. 

0.78 

0.44 

0.26 

Wind  speed 

0.62 

0.48 

0.59 

u 

0.38 

0.18 

0.27 

V 

0.34 

0.32 

0.33 

Figure  4a.  Scatter  diagram  of  model  calculation  versus  observation  of 
temperature,  Colorado  model  domain. 


Figure  4b.  Scatter  diagram  of  model  calculation  vs.  dew  point  temperature, 
Colorado  model  domain. 


Figure  4c.  Scatter  diagram  of  model  calculation  versus  wind  vector  components  u  (left-hand  side)  and 
v  (right-hand  side),  Colorado  model  domain. 
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Figure  4d.  Scatter  diagram  of  model  calculation  versus  wind  speed, 
Colorado  model  domain. 


Figure  5a.  Scatter  diagram  of  model  calculation  versus  observation  of 
temperature,  Washington  model  domain. 
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Figure  5b.  Scatter  diagram  of  model  calculation  vs.  dew  point  temperature, 
Washington  model  domain. 


Figure  5c.  Scatter  diagram  of  model  calculation  versus  wind  vector  components,  u  (left-hand  side) 
and  v  (right-hand  side),  Washington  model  domain. 
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Figure  5d.  Scatter  diagram  of  model  calculation  versus  wind  speed, 
Washington  model  domain. 


Figure  6a.  Scatter  diagram  of  model  calculation  versus  observation  of 
temperature,  Florida  model  domain. 
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Figure  6b.  Scatter  diagram  of  model  calculation  versus  dew  point 
temperature,  Florida  model  domain. 


Figure  6c.  Scatter  diagram  of  model  calculation  versus  wind  vector  components  u  (left-hand  side)  and 
v  (right-hand  side),  Florida  model  domain. 
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Figure  6d.  Scatter  diagram  of  model  calculation  versus  wind  speed,  Florida 
model  domain. 

The  ranges  of  variation  of  temperature  and  dew  point  temperature  for  the 
Florida  domain  are  the  narrowest  among  the  three  domains,  reflecting  the 
maritime  climate,  and  those  for  the  Colorado  domains  are  the  largest, 
reflecting  the  continental  climate. 

For  temperature  and  dew  point  temperature,  the  correlation  coefficients  for 
the  Colorado  model  domain  are  the  largest,  and  those  for  the  Washington 
domain  follow.  It  is  encouraging  to  see  that  the  highest  correlation  values 
for  temperature  and  dew  point  temperature  are  produced  for  the  Colorado 
model  domain  by  the  BFM  despite  the  large  range  of  values. 

For  wind  vector  components  u  and  v,  and  speed,  the  correlation  coefficients 
are  smaller  than  those  for  temperature  and  dew  point  temperature.  The 
wind  speed  scatter  diagram  for  the  Colorado  domain  is  the  most  scattered 
of  the  three  domains,  again  reflecting  the  complex  terrain.  On  the  other 
hand,  the  wind  speed  scatter  diagram  for  the  Florida  domain  shows  much 
less  scattering  distribution  than  the  diagrams  for  the  other  two  domains. 
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4.2  Time  Series  of  the  Statistical  Parameters 


MD,  AD,  RMSE,  and  RMSVE  are  calculated  at  the  forecast  periods  of  0, 3, 6, 9, 
12, 15, 21  and  24  h.  Time  0  h  corresponds  to  12  UTC.  Forecast  calculations  are 
performed  three  times  with  three  different  initial  conditions.  In  figures  7 
through  9,  three  curves  are  drawn: 

K=1  represents  the  statistics  of  the  BFM  initialized  with  NOGAPS  +  upper  air 
+  surface  data, 

K=2  represents  those  initialized  with  NOGAPS  +  upper-air  data,  and 
K=3  represents  those  initialized  with  NOGAPS  data  only. 

In  the  figures,  only  MD  and  AD  for  all  the  meteorological  variables,  and 
RMSVEs  for  wind  vector  are  shown  in  the  figures.  The  RMSEs  are  calculated 
but  are  not  shown  in  this  report,  because  the  RMSE  usually  varies 
proportionally  to  the  AD. 

4.2.1  Temperature 

Figures  7,  8,  and  9  are  the  MD  and  AD  of  temperature  for  Colorado, 
Washington,  and  Florida,  respectively.  It  is  clear  from  these  three  figures  that 
the  BFM  initialized  with  NOGAPS  +  upper-air  data  (K=l  and  K=2)  produced 
better  forecast  than  that  with  NOGAPS  data  only  (K=3). 

For  the  Colorado  model  domain,  the  initial  data  without  upper-air  data 
produced  worse  temperature  forecast  than  those  with  upper-air  data.  The 
effect  of  using  the  surface  data  is  seen  at  the  forecast  periods  between  0  and 
3  h.  For  very  short-term  forecast  up  to  3  h,  the  nudging  of  the  surface  data 
produces  improved  forecasts  of  surface  temperature.  After  3  h,  the  curves 
with  K=1  and  K=2  are  very  similar,  implying  that  the  effect  of  surface 
temperature  nudging  becomes  insignificant.  The  curves  of  MD  indicate  that 
during  daytime  hours  the  temperatures  forecasted  by  the  BFM  are  lower  than 
observed  over  the  Colorado  model  domain. 

For  the  Washington  model  domain,  the  use  of  upper-air  data  for  the  initial 
condition  produced  better  temperature  forecasts,  but  the  influence  of  upper-air 
data  is  not  as  significant  as  over  the  Colorado  model  domain.  The  effect  of 
surface  temperature  nudging  in  initialization  is  seen  again  in  the  first  3  h  of 
forecast.  There  are  no  significant  differences  between  the  three  MD  curves, 
and  the  values  of  all  three  curves  for  K=l,  2,  and  3  stayed  around  0°  C, 
indicating  there  is  no  significant  bias.  For  the  Florida  model  domain,  similar 
to  the  results  from  the  Colorado  and  Washington  domains,  the  initial  condition 
without  upper-air  data  produced  inferior  forecast  of  temperature  to  those  with 
upper-air  data  included.  MD  curves  show  that  the  surface  temperature  over 


the  land  of  the  Florida  model  domain  was  underpredicted  during  daytime 
hours  and  overpredicted  during  nighttime  hours.  The  amplitude  of  daily 
temperature  cycle  forecasted  by  the  BFM  was  not  large  enough  to  match  that 
observed. 


Figure  7.  Time  series  of  MD  and  AD  of  temperature,  Colorado  model  domain. 

Note: 

K=l,  for  initialization  with  NOGAPS  +  upper  air  +  surface  data, 

K=2,  for  initialization  with  NOGAPS  +  upper-air  data,  and 
K=3,  for  initialization  with  NOGAPS  data. 
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Figure  8.  Time  series  of  MD  and  AD  of  temperature,  Washington  model  domain. 

Note: 

K=l,  for  initialization  with  NOGAPS  +  upper  air  +  surface  data, 

K=2,  for  initialization  with  NOGAPS  +  upper-air  data,  and 
K=3,  for  initialization  with  NOGAPS  data. 


Figure  9.  Time  series  of  MD  and  AD  of  temperature,  Florida  model  domain. 

Note: 

K=l,  for  initialization  with  NOGAPS  +  upper  air  +  surface  data, 

K=2,  for  initialization  with  NOGAPS  +  upper-air  data,  and 
K=3,  for  initialization  with  NOGAPS  data. 

* 
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4.2.2  Dew  Point  Temperature 

Figures  10, 11,  and  12  are  the  plotting  of  MD  and  AD  time  series  for  the  three 
model  domains.  For  the  Colorado  model  domain  (figure  10),  the  model 
forecast  of  dew  point  temperature  averages  more  than  3°  off  from  the 
observations.  Among  the  three  model  domains,  the  AD  values  for  the 
Colorado  model  domain  are  the  largest.  For  daylight  hours,  the  model 
forecasted  the  dew  point  temperature  lower  than  observed.  This  is  particularly 
significant  with  K=1  and  2,  which  represent  the  initial  conditions  with  upper- 
air  data.  It  is  not  clear  why  the  initial  conditions  with  upper-air  data  generated 
inferior  forecasts  of  dew  point  temperature  to  those  only  with  NOGAPS  data. 


Dew  Pt  TemplC) 


Figure  10.  Time  series  of  MD  and  AD  of  dew  point  temperature,  Colorado  model 
domain. 


Figure  11.  Time  series  of  MD  and  AD  of  dew  point  temperature,  Washington  model 
domain. 

For  the  Washington  model  domain,  the  model  generated  lower  dew  point 
temperatures  than  observed  throughout  24-h  forecast  period.  Similar  to  the 
Colorado  model  domain,  the  initial  conditions  with  upper-air  data  produced 
lower  dew  point  temperatures  than  those  without  upper-air  data. 
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Dew  Pt  Temp l C ) 


Figure  12.  Time  series  of  MD  and  AD  of  dew  point  temperature,  Florida  model 
domain. 

For  the  Florida  model  domain,  the  model  produced  greater  dew  point 
temperatures  than  observed  throughout  the  24-h  forecast  period,  except  for  the 
0  and  3-h  periods.  The  AD  values  are  the  smallest  among  the  three  model 
domains  throughout  the  24-h  forecast  period.  There  are  no  significant 
differences  among  the  three  forecasts  with  different  initial  conditions. 

From  the  inspection  of  the  moisture  values  of  NOGAPS  data,  which  is  used  for 
initial  and  boundary  values  of  the  BFM,  it  is  suspected  that  NOGAPS  tends  to 
forecast  drier  atmosphere  than  observed.  Therefore,  the  dew  point 
temperatures  by  the  NOGAPS  are  compared  to  the  observed  values. 

Tables  3, 4,  and  5  show  MD  and  AD  between  NOGAPS  data  and  observations 
for  dew  point  temperature,  respectively,  for  the  Colorado,  Washington,  and 
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Florida  model  domains.  The  NOGAPS  data  used  for  the  initialization  and 
boundary  conditions  of  the  BFM  at  0,  12,  and  24  h  are  compared  with  the 
surface  data. 


Table  3.  MD  and  AD  at  forecast  periods  of  0, 12,  and  24  h  between  NOGAPS  data 
and  observation,  Colorado  model  domain _ 


Forecast  horns 

MD 

AD 

Oh 

-2.2 

3.6 

12  h 

-1.2 

3.5 

24  h 

-1 

3.5 

Table  4.  MD  and  AD  at  forecast  periods  of  0, 12,  and  24  h  between  NOGAPS  data 
and  observation  for  Washington  model  domain  _ 


Forecast  hours 

MD 

AD 

Oh 

-0.9 

1.8 

12  h 

-0.2 

2 

24  h 

-1.3 

2 

Table  5.  MD  and  AD  at  forecast  periods  of  0, 12,  and  24  h  between  NOGAPS  data 
and  observation  for  Florida  model  domain _ 


Forecast  hours 

MD 

AD 

Oh 

-0.7 

1.2 

12  h 

0 

1.2 

24  h 

0.8 

1.3 

From  these  tables,  the  MD  indicates  that  dew  point  temperature  fields  of 
NOGAPS  are  lower  than  those  observed  throughout  the  24-h  forecast  periods 
over  the  Colorado  and  Washington  domains.  For  the  Florida  domain,  they  are 
lower  than  observed  in  the  first  12  h.  However,  the  magnitudes  of  the  MD  and 
AD  between  NOGAPS  and  observation  at  0  h  given  in  tables  3  and  4  are  smaller 
than  those  shown  in  figures  10  and  11.  Therefore,  it  is  suspected  that  in  the 
Colorado  and  Washington  domains,  the  BFM's  prognostic  calculations  might 
have  numerically  produced  lower  dew  point  temperatures  than  observed 
during  the  early  stages  of  forecast  periods. 
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4.2.3  Wind  Speed  and  Wind  Vector 

Unlike  temperature  and  dew  point  temperature,  there  are  no  significant 
differences  in  MD,  AD  and  RMSVE  among  the  three  different  initialization 
methods  for  all  three  model  domains.  The  values  of  MD  are  between  0  and 
1  m/s  for  the  three  domains  throughout  the  24-h  forecast  period,  implying  that 
the  BFM  calculates  the  wind  speeds  slightly  greater  than  those  of  observations. 
For  all  three  domains,  AD  values  are  also  between  0  and  1  m/sec.  For  wind 
speed  forecasts,  there  are  no  significant  differences  among  three  different  model 
domains.  The  AD  values  less  than  1  m/  sec  are  well  within  the  weather  forecast 
data  criteria  used  by  the  U.S.  Air  Force,  which  requires  that  for  observed  wind 
speed  less  than  10  m/sec,  AD  is  to  be  within  1  m/ s,  and  for  observed  wind 
speed  greater  than  10  m/sec,  AD  is  to  be  smaller  than  2.5  m/sec.  [5] 

For  wind  vector  forecasts,  the  RMSVE  values  for  the  Colorado  model  domain 
shows  the  greatest  values  among  the  three  model  domains,  and  those  for  the 
Washington  model  domain  show  the  smallest. 

In  the  comparison  and  evaluation  study  of  the  MM5  and  the  BFM  over  the 
WSMR  with  unit  grid  spacing  of  3.33  km,  the  AD  values  for  wind  speed  are 
between  2  and  3  m/sec  for  both  MM5  and  BFM.  In  a  similar  study  over  the 
National  Training  Center  area  in  which  BFM  was  run  with  unit  grid  spacing  of 
2.5  km,  the  AD  values  for  wind  speed  were  also  between  2  and  3  m/sec.  For 
both  studies,  the  AD  for  wind  speed  are  significantly  greater  than  those  for  the 
current  three  model  domains,  which  were  run  with  unit  grid  spacing  of  10  km. 
At  this  time,  why  the  current  study  shows  better  results  in  wind  speed  forecast 
than  these  two  studies  is  unknown.  Further  examination  is  needed  to  find  the 
exact  reasons  for  the  differences  and  a  few  items  to  consider  that  are: 

1.  model  performances  in  different  grid  spacing, 

2.  quality  of  observed  data,  and 

3.  model  performances  in  different  seasons.  [1,6] 

4.3  Comparison  of  the  BFM  with  NOGAPS  and  NORAPS 

The  forecast  results  of  the  BFM  are  compared  with  those  given  by  the  NOGAPS 
and  the  NORAPS.  For  the  NOGAPS  and  the  NORAPS,  data  are  given  every  1  ° 
and  0.5°,  respectively,  and  are  interpolated  bilinearly  to  the  observation 
locations. 
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4.3.1  Temperature 

Figures  16, 17,  and  18  show  the  comparisons  of  temperature  forecasts  for  the 
Colorado,  Washington,  and  Florida  model  domains,  respectively.  The  values  of 
RMSE  and  AD  are  compared  in  the  form  of  bar  diagrams.  In  these  figures,  the 
comparisons  for  the  forecast  periods  of  0, 6,12,  and  24  h  are  shown  in  (a),  (b),  (c), 
and  (d),  respectively.  The  bar  diagram  at  the  left-hand  side  is  for  the  RMSE,  and 
that  at  the  right-hand  side  is  for  the  AD.  In  these  figures,  three  BFM  forecasts 
with  the  different  initial  conditions  (see  section  3.2)  are  shown,  marked  as  BFM1, 
BFM2,  and  BFM3. 

For  all  the  model  domains,  the  initial  temperature  fields  predicted  by  the  BFM 
are  calculated  best  when  the  surface  observed  data  are  used,  but  at  the  forecast 
period  of  6  h,  the  influences  of  use  of  surface  temperature  disappears.  For 
Colorado  and  Washington  model  domains,  where  the  terrain  is  more  complex 
than  Florida,  the  use  of  the  BFM  improves  the  temperature  field  forecast. 
However,  for  Horida  model  domain,  the  temperature  fields  forecast  by  the  BFM 
do  not  significantly  improve  the  comparison  statistics  over  the  forecasts  by 
NOGAPS  and  NORAPS. 

4.3.2  Wind  Speed  and  Vector 

Figures  19,  20,  and  21  are  the  comparisons  of  wind  speed  and  vectors, 
respectively,  for  the  Colorado,  Washington,  and  Florida  model  domains.  In 
these  figures,  the  RMSE  is  shown  at  the  left-hand  side,  the  AD  at  the  middle,  and 
the  RMSVE  at  the  right-hand  side.  For  the  Colorado  and  Washington  model 
domains,  the  BFM  calculations  produced  better  wind  fields  than  NOGAPS  and 
NORAPS  throughout  the  24-h  forecast  period,  but  for  the  Florida  model  domain, 
the  BFM  does  not  improve  the  statistics  over  NOGAPS  and  NORAPS.  A  recent 
study  by  Dumais  clearly  shows  that  the  ability  of  BFM  to  forecast  local  diurnal 
winds  is  largely  determined  by  its  horizontal  resolution  and  treatment  of  the 
local  surface  feature  data.  In  the  present  study,  the  horizontal  grid  spacing  is  10 
km,  and  constant  values  for  surface  albedo  and  soil  heat  conductivity  are  used 
over  the  land.  Rao  et  al.  for  realistic  simulation  of  sea  breeze  circulation,  the 
horizontal  resolutions  of  ~100  m  for  terrain  elevation,  and  surface  features  such 
as  soil  type  and  soil  moisture  are  needed.  It  is,  therefore,  suspected  that  the 
present  forecast  calculations  by  the  BFM  for  the  Florida  domain  had  failed  to 
simulate  the  effects  of  sea-land  breeze  circulation.  BFM  showed  better  forecast 
skills  for  predictions  of  wind  fields  over  complex  terrain  than  NOGAPS  and 
NORAPS  [7,8] 
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Figures  16(c)  and  (d).  Comparison  of  BFM  temperature  forecast  performance  to 
NOGAPS  and  NORAPS,  the  Colorado  model  domain. 

Note:  (c)  12  h,  and  (d)  24-h  forecast  period.  Left-hand  side:  RMSE,  right-hand  side:  AD. 
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Figures  17(a)  and  (b).  Comparison  of  BFM  temperature  forecast  performance  to 
NOGAPS  and  NORAPS,  Washington  model  domain. 


Note:  (a)  0  h,  and  (b)  6-h  forecast  period.  Left-hand  side:  RMSE,  right-hand  side:  AD. 
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Figures  17(c)  and  (d).  Comparison  of  BFM  temperature  forecast  performance  to 
NOGAPS  and  NORAPS,  Washington  model  domain. 


Note:  (c)  12  h,  and  (d)  24-h  forecast  period.  Left-hand  side:  RMSE,  right-hand  side:  AD. 
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Figures  18(a)  and  (b).  Comparison  of  BFM  temperature  forecast  performance  to 
NOGAPS  and  NORAPS,  Florida  model  domain. 

Note:  (a)  0  h,  and  (b)  6-h  forecast  period.  Left-hand  side:  RMSE,  right-hand  side:  AD. 
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Figures  18(c)  and  (d).  Comparison  of  BFM  temperature  forecast  performance 
NOGAPS  and  NORAPS,  Florida  model  domain. 

Note:  (c)  12  h,  and  (d)  24-h  forecast  period.  Left-hand  side:  RMSE,  right-hand  side:  AD 
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Figures  19(a)  and  (b).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS,  for  wind  speed  and 
vectors,  Colorado  model  domain. 

Note:  (a)  0  h,  (b)  6-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand  side: 
RMSVE. 


BFM1:  initialized  with  NOGAPS  +  upper  air  +  surface  data, 
BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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Figures  19(c)  and  (d).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS,  for  wind  speed  and 
vectors,  Colorado  model  domain. 

Note:  (c)  12  h,  (d)  12-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand  side: 
RMSVE. 

BFM1:  initialized  with  NOGAPS  +  upper  air  +  surface  data, 

BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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Figures  20(a)  and  (b).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS  for  wind  speed 
and  vectors  for  Washington  model  domain^ 

Note:  (a)  0  h,  (b)  6-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand 
side:  RMSVE. 

BFM1:  initialized  with  NOGAPS  +  upper  air  +  surface  data, 

BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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Figures  20(c)  and  (d).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS  for  wind  speed 
and  vectors,  Washington  model  domain. 

Note:  (c)  12  h,  (d)  12-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand 
side:  RMSVE. 

BFM1:  initialized  with  NOGAPS  +upper  air  +  surface  data, 

BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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Figures  21(a)  and  (b).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS  for  wind  speed 
and  vectors,  Florida  model  domain. 

Note:  (a)  0  h,  (b)  6-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand 
side:  RMSVE. 

BFM1:  initialized  with  NOGAPS  +upper  air  +  surface  data, 

BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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Figures  21(c)  and  (d).  Comparison  of  BFM  performance  to  NOGAPS  and  NORAPS,  wind  speed 
and  vectors,  Florida  model  domain. 

Note:  (c)  12  h,  (d)  12-h  forecast  periods.  Left-hand  side:  RMSE  of  speed.  Center:  AD  of  speed,  right-hand 
side:  RMSVE. 

BFM1:  initialized  with  NOGAPS  +upper  air  +  surface  data, 

BFM2:  initialized  with  NOGAPS  +  upper-air  data,  and 
BFM3:  initialized  with  NOGAPS. 
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5.0  Summary 

BFM  forecast  calculations  were  made  over  the  different  climatological  and 
topographical  model  domains  of  Colorado,  Washington,  and  Florida. 
Forecast  calculations  were  made  for  a  30-day  period  from  June  23  to  July 
22,  1998.  Three  different  initial  conditions  were  generated  from  the 
combination  of: 

(1)  NOGAPS  +  upper  air  +  surface  observation  data, 

(2)  NOGAPS  +  upper  air,  and 

(3)  NOGAPS  only. 

1.  Scatter  diagrams  of  forecast  versus  observed  data  showed  the  forecast 
data  of  temperature,  dew  point  temperature,  and  wind  speed  and  vector 
components  u  and  v  over  the  Colorado  model  domain  correlated  better  with 
observations  than  those  over  Washington  and  Florida.  The  reasons  why  the 
Florida  model  domain  did  not  produce  better  correlations  than  the  other  two 
model  domains  are  not  obvious.  The  Florida  domain  is  much  flatter  than  the 
Colorado  and  W ashington  domains,  which  implies  simpler  forecast  problem. 
On  the  other  hand,  the  Florida  domain  is  surrounded  by  the  sea.  In  the  BFM 
on  the  1METS,  the  sea  surface  temperature  distribution  is  provided  by  the 
monthly  average  sea  surface  temperature  data  over  the  globe  obtained  from 
the  U.  S.  Air  Force  Combat  Climatological  Center  in  1994.  To  improve  the 
temperature  and  dew  point  temperature  distributions  over  the  model 
domains  surrounded  by  the  sea,  the  daily  sea  surface  temperature 
distribution  may  be  needed.  In  the  near  future,  the  earth  surface  skin 
temperature  by  NOGAPS,  which  provides  daily  sea  surface  temperature 
based  on  satellite  and  buoy  observations,  will  be  incorporated  in  the  BFM 
operational  on  the  IMETS  system. 

2.  For  all  three  model  domains,  the  forecast  temperature  fields  initialized 
with  (1)  and  (2)  were  statistically  better  than  those  initialized  with  (3).  For 
the  Colorado  and  Florida  domains,  the  model  produced  surface 
temperatures  lower/higher  than  observation  during  daytime /nighttime, 
indicating  the  daily  amplitudes  of  temperature  predicted  by  the  model  were 
smaller  than  what  actually  occurred.  For  the  Washington  model  domain,  no 
significant  bias  of  temperature  was  found.  The  AD  values  for  the  24-h 
forecast  period  for  all  three  domains  varied  between  2°  and  3°  C  for  the 
BFM  calculations  using  the  initial  conditions  of  (1)  and  (2).  Forecast  data 
accuracy  criteria  used  by  the  U.S.  Air  Force  for  temperature  is  2°  C. 
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Therefore,  the  BFM  produced  slightly  worse  temperature  values  than 
those  required  by  the  U.S.  Air  Force  criteria.  [1] 

3.  For  the  Colorado  and  Washington  domains,  MD  values  of  dew  point 
temperature  were  negative  during  most  of  24-h  forecast  periods.  For  the 
Colorado  domain,  this  was  more  significant  when  the  model  was  initialized 
with  (1)  and  (2)  in  the  above  than  when  initialized  with  (3).  The  initial  fields 
created  by  compositing  NOGAPS  data  with  upper-air  data  might  have 
caused  this  numerical  drying.  Further  studies  are  needed  to  find  the  exact 
reasons  for  this  shortcoming  of  the  BFM.  For  the  Florida  model  domain,  on 
the  other  hand,  the  values  of  MD  for  dew  point  temperature  stayed  positive 
throughout  the  24-h  period. 

4.  For  wind  fields,  the  three  different  data  initializations  combinations  did 
not  produce  significantly  different  BFM  wind  fields  throughout  the  24-h 
forecast  periods.  There  was  little  difference  in  the  values  of  MD,  AD  and 
RMSVE  among  three  different  initialization  methods  for  all  three  model 
domains.  The  values  of  MD  for  wind  speed  are  between  0  and  1  m/sec,  and 
the  values  of  AD  are  also  between  0  and  1  m/sec  for  the  three  model 
domains  throughout  24-h  forecast  period.  For  wind  vector  forecasts,  the 
values  of  RMSVE  for  Colorado  domain  were  the  greatest  among  the  three 
model  domains,  and  those  for  the  Washington  domains  were  the  smallest. 

5.  For  the  Colorado  and  Washington  model  domains  where  the  terrain  is 
more  complex  than  Florida,  the  use  of  the  BFM  improves  temperature  field 
forecast  over  those  of  NOGAPS  and  NORAPS.  For  the  Florida  model 
domain,  there  was  not  significant  improvement  in  the  temperature  field 
forecast  by  the  BFM  over  those  of  NOGAPS  and  NORAPS. 

6.  BFM  calculations  produced  better  wind  fields  than  NOGAPS  and 
NORAPS  throughout  the  24-h  forecast  periods  for  the  Colorado  and 
Washington  model  domains,  but  the  BFM  for  the  Florida  domain,  did  not 
produce  improved  wind  fields  over  those  by  NOGAPS  and  NORAPS.  For 
the  Florida  domain,  horizontal  grid  resolution  of  10  km  might  have  been  too 
coarse  to  simulate  sea  and  land  breeze  circulation. 

Although  the  present  study  has  shown  some  limitations  of  the  BFM,  the 
BFM  is  a  valuable  and  useful  tool  for  the  short-range  weather  forecast  over 
complex  terrains  when  used  with  initial  and  time-dependent  lateral  values 
provided  by  a  global  scale  forecast  model  such  as  NOGAPS  and  other 
available  meteorological  data. 
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Acronyms 


AD 

absolute  difference 

ARL 

U.S.  Army  Research  Laboratory 

BFM 

Battlescale  Forecast  Model 

CAAM 

Computer  Assisted  Artillery  Meteorology 

HOTMAC 

Higher  Order  Turbulence  Model  for  Atmospheric  Circulation 

IMETS 

Integrated  Meteorological  System 

MD 

mean  difference 

MEL 

Master  Environmental  Library 

NOGAPS 

Navy  Operational  Global  Atmospheric  Prediction  System 

NORAPS 

Navy  Operational  Regional  Atmospheric  Prediction  System 

RMSE 

root  mean  square  error 

RMSVE 

root  mean  square  vector  error 

WSMR 

White  Sands  Missile  Range 

MM5 

Fifth-Generation  National  Center  for  Atmospheric  Research/ 
Penn  State  Mesoscale  Model 
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ARMY  RESEARCH  LABORATORY 
AMSRL  Cl  E 
COMP  &  INFO  SCI  DIR 
WSMRNM  88002-5501 

DTIC 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FT  BEL  VOIR  VA  22060-6218 

US  ARMY  MISSILE  CMND 
AMSMI 

REDSTONE  ARSENAL  AL  35898-5243 

US  ARMY  DUGWAY  PROVING  GRD 
STEDP3 

DUGWAY  UT  84022-5000 

USTRADOC 
ATCD  FA 

FT  MONROE  VA  23651-5170 

WSMR  TECH  LIBRARY  BR 
STEWS  IM  IT 
WSMRNM  88002 

US  ARMY  RESEARCH  LAB 
AMSRL  D  DR  D  SMITH 
2800  POWDER  MILL  RD 
ADELPHI MD  20783-1197 

US  ARMY  CECOM 
INFORMATION  &  INTELLIGENCE 
WARFARE  DIRECTORATE 
ATTN  AMSEL  RD  IW  IP 
FORT  MONMOUTH  NJ  07703-521 1 

ARMY  RESEARCH  LABORATORY 

AMSRL  Cl  EW 

ATTN  MR  HEMNI 

COMP  &  INFO  SCIENCES  DIR 

WSMRNM  88002-5501 


Record  copy 


TOTAL 


